So if I understand, there's no post data descriptor for zip64 but the central directory has the values?

Then you can't use streaming reads on zip64?

This pretty shitty. I guess they updated APPNOTE.txt:

4.3.9.5 An extensible data descriptor will be released in a 
      future version of this APPNOTE.  This new record is intended to
      resolve conflicts with the use of this record going forward,
      and to provide better support for streamed file processing.

      4.3.9.6 When the Central Directory Encryption method is used, 
      the data descriptor record is not required, but MAY be used.  
      If present, and bit 3 of the general purpose bit field is set to 
      indicate its presence, the values in fields of the data descriptor
      record MUST be set to binary zeros.  See the section on the Strong 
      Encryption Specification for information. Refer to the section in 
      this document entitled "Incorporating PKWARE Proprietary Technology 
      into Your Product" for more information.

So there's currently no way to forward-only read entries out of a zip64.

Would a good test to just be forcing zip64 on a small file? The values getting overloaded shouldn't really matter.

Streaming

Yes, that is my understanding: streaming cannot write 64bit values in the post-data.

I know they previously said that the post-data header should have 8-byte values for zip64, but they forgot to mention how to figure out if the file is zip64 when reading forward-only. The update you mention is probably a thought that there should be some kind of identifier in the post-data region.

You can still use forward-only reading, but you cannot verify if the lengths are correct, if the stream is larger than 4GiB.

Edit: This problem is worst if you write the archive forward-only AND read the archive in forward-only. If the stream is seekable, it is possible to forgo the post-data and use the zip64 extras, such that forward-only reading is possible.

Testing

You can force zip64 to have the extra headers, but I would expect errors to show up when values are hitting the unit.MaxValue limit.

Misc

One detail I forgot to mention in the initial text is that the list of files report the Count as int, so there is an overflow there if you have more than int.MaxValue files in the archive. SharpCompress does not use this value for anything, but other libraries may.

You could always just say upfront to write everything as zip64 and deal with streaming that way. Then write post descriptors with 8 bytes. I guess that would be incompatible with other implementations?

I guess if the code allows forward-only reading but you'd have to read the central directory for the valid length that's something. However, that's now how the reader works now. The central directory isn't used with readers, only archives. Ugh.

I guess the count should be 'ulong' like everything else.

This code really needs some refactor lovin too I think.

Streaming

I guess we can look at "what others do", but that is likely to cause errors. If the readers expect 4-byte values, there should at least be some indicator somewhere that the values are not 4-byte.

Count overflow

Yes, but that would mean replacing the system List<T>, which is a rather intrusive change.

I usually test with WinRAR and 7Zip myself.

I guess I see the point about the collections. Let's save that for a rainy day.

Maybe a better approach is to simply throw an exception if that case is hit?

If the stream is non-seekable, and the input data is larger than uint.MaxValue, we could throw an exception.

Yes. Explicit exceptions are good if there's no fix. At least to let people know's happening.

Thanks again!

Should we do the same for the case where we use a seek-able stream, and do not have space for the zip64 extra fields?

Maybe we could then choose to not make space for it by default (remove my 2GiB threshold) and make the caller set the zip64 flag when creating the archive or stream exceeds 4GiB ?

I would rather be explicit in these cases and force the user to know what they're doing. Especially when getting into the zip64 file size area.

I guess just measure then throw exceptions if it's not zip64 mode.

Ok, I will update the code to require explicit zip64 activation when writing streams larger than 4GiB. And disallow writing streams larger than 4GiB in forward-only mode.

Archives larger than 4GiB can be transparently supported, as long as all streams inside are less than 4GiB individually.

I have updated the logic to require setting the zip64 flag before writing a stream larger than 4GiB.
I have also disabled zip64 support for writing non-seekable streams.

I have added the check to the Write method of the ZipWritingStream to avoid throwing exceptions in the Dispose method.

Not sure how you prefer this to work, but we cannot know in advance if the write expands the file beyond 4GiB, so I have made a pre-check and a post-check. If we catch it in the pre-check, the file is correct (but with a shorter stream), if we catch it in the post check, invalid data is now in the archive.
Maybe the post-check is sufficient? Is it acceptable to close the base stream if the write has produced an invalid file?

I have also added a unittest for trying out various parts of the zip64 features.

It looks like you throw an exception when the file produced will be invalid so that seems sufficient.